Blind source separation for non-stationary mixing

The original publication is available at www.springerlink.comBlind source separation attempts to recover independent sources which have been linearly mixed to produce observations. We consider blind source separation with non-stationary mixing, but stationary sources. The linear mixing of the independent sources is modelled as evolving according to a Markov process, and a method for tracking the mixing and simultaneously inferring the sources is presented. Observational noise is included in the model. The technique may be used for online filtering or retrospective smoothing. The tracking of mixtures of temporally correlated is examined and sampling from within a sliding window is shown to be effective for destroying temporal correlations. The method is illustrated with numerical examples

Robust autoregression: Student-t innovations using variational Bayes

Copyright © 2011 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Autoregression (AR) is a tool commonly used to understand and predict time series data. Traditionally the excitation noise is modelled as a Gaussian. However, real-world data may not be Gaussian in nature, and it is known that Gaussian models are adversely affected by the presence of outliers. We introduce a Bayesian AR model in which the excitation noise is assumed to be Student-t distributed. Variational Bayesian approximations to the posterior distributions of the model parameters are used to overcome the intractable integrations inherent in the Bayesian model. Independent automatic relevance determination (ARD) priors over each of the AR coefficients are used to estimate the model order. Using synthetic data, we show that the Student-t model performs well against both Gaussian and leptokurtic data, in terms of parameter estimation (including the model order) and is much more robust to outliers than either Gaussian or finite mixtures of Gaussian models. We apply the model to strongly leptokurtic EEG signals and show that the Student-t model makes more accurate one-step-ahead predictions than the Gaussian model and provides more consistent estimates of the AR coefficients over simultaneously recorded EEG channels

Inferring the eigenvalues of covariance matrices from limited, noisy data

ArticleCopyright © 2000 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.The eigenvalue spectrum of covariance matrices is of central importance to a number of data analysis techniques, Usually, the sample covariance matrix is constructed from a limited number of noisy samples, We describe a method of inferring the true eigenvalue spectrum from the sample spectrum. Results of Silverstein, which characterize the eigenvalue spectrum of the noise covariance matrix, and inequalities between the eigenvalues of Hermitian matrices are used to infer probability densities for the eigenvalues of the noise-free covariance matrix, using Bayesian inference. Posterior densities for each eigenvalue are obtained, which yield error estimates. The evidence framework gives estimates of the noise variance anal permits model order selection by estimating the rank of the covariance matrix, The method is illustrated with numerical examples

Bayesian estimation and classification with incomplete data using mixture models

©2004 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.Reasoning from data in practical problems is frequently hampered by missing observations. Mixture models provide a powerful general semi-parametric method for modelling densities and have close links to radial basis function neural networks (RBFs). We extend the Data Augmentation (DA) technique for multiple imputation to Gaussian mixture models to permit fully Bayesian inference of model parameters and estimation of the missing values. The method is compared to imputation using a single normal density on synthetic and real-world data. In addition to a lower mean squared error than can be achieved by simple imputation methods, mixture Models provide valuable information on the potentially multi-modal nature of imputed values. The DA formalism is extended to a classifier closely related to RBF networks permitting Bayesian classification with incomplete data; the technique is illustrated on synthetic and real datasets

ROC Optimisation of Safety Related Systems

1st Workshop on ROC Analysis in Artificial Intelligence (ROCAI 2004), part of the 16th European Conference on Artificial Intelligence, Valencia, Spain, 22-27 August 2004Many safety related and critical systems warn of potentially dangerous events; for example the Short Term Conflict Alert (STCA) system warns of airspace infractions between aircraft. Although installed with current technology such critical systems may become out of date due to changes in the circumstances in which they function, operational procedures and the regulatory environment. Current practice is to ‘tune’ by hand the many parameters governing the system in order to optimise the operating point in terms of the true positive and false positive rates, which are frequently associated with highly imbalanced costs. In this paper we cast the tuning of critical systems as a multiobjective optimisation problem. We show how a region of the optimal receiver operating characteristic (ROC) curve may be obtained, permitting the system operators to select the operating point. We apply this methodology to the STCA system, using a multi-objective (1 + 1)-evolution strategy, showing that we can improve upon the current hand-tuned operating point as well as providing the salient ROC curve describing the true-positive versus false-positive tradeoff

A Bayesian Framework for Active Learning

Copyright © 2010 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.We describe a Bayesian framework for active learning for non-separable data, which incorporates a query density to explicitly model how new data is to be sampled. The model makes no assumption of independence between queried data-points; rather it updates model parameters on the basis of both observations and how those observations were sampled. A `hypothetical' look-ahead is employed to evaluate expected cost in the next time-step. We show the efficacy of this algorithm on the probabilistic high-low game which is a non-separable generalisation of the separable high-low game introduced by Seung et al. Our results indicate that the active Bayes algorithm performs significantly better than passive learning even when the overlap region is wide, covering over 30% of the feature space

Multi-objective optimisation in the presence of uncertainty

2005 IEEE Congress on Evolutionary Computation, Edinburgh, Scotland, 2-5 September 2005The codebase for this paper is available at https://github.com/fieldsend/ieee_cec_2005_bayes_uncertainThere has been only limited discussion on the effect of uncertainty and noise in multi-objective optimisation problems and how to deal with it. Here we address this problem by assessing the probability of dominance and maintaining an archive of solutions which are, with some known probability, mutually non-dominating.We examine methods for estimating the probability of dominance. These depend crucially on estimating the effective noise variance and we introduce a novel method of learning the variance during optimisation.Probabilistic domination contours are presented as a method for conveying the confidence that may be placed in objectives that are optimised in the presence of uncertainty

The Rolling Tide Evolutionary Algorithm: A Multi-Objective Optimiser for Noisy Optimisation Problems

As the methods for evolutionary multiobjective optimization (EMO) mature and are applied to a greater number of real-world problems, there has been gathering interest in the effect of uncertainty and noise on multiobjective optimization, specifically how algorithms are affected by it, how to mitigate its effects, and whether some optimizers are better suited to dealing with it than others. Here we address the problem of uncertain evaluation, in which the uncertainty can be modeled as an additive noise in objective space. We develop a novel algorithm, the rolling tide evolutionary algorithm (RTEA), which progressively improves the accuracy of its estimated Pareto set, while simultaneously driving the front toward the true Pareto front. It can cope with noise whose characteristics change as a function of location (both design and objective), or which alter during the course of an optimization. Four state-of-the-art noise-tolerant EMO algorithms, as well as four widely used standard EMO algorithms, are compared to RTEA on 70 instances of ten continuous space test problems from the CEC'09 multiobjective optimization test suite. Different instances of these problems are generated by modifying them to exhibit different types and intensities of noise. RTEA seems to provide competitive performance across both the range of test problems used and noise types

Multi-objective optimisation of safety related systems: An application to Short Term Conflict Alert.

Copyright © 2006 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.Notes: In this paper multi-objective optimisation is used for the first time to adjust the 1500 parameters of Short-Term Conflict Alert systems to optimise the Receiver Operating Characteristic (ROC) by simultaneously reducing the false positive rate and increasing the true positive alert rate, something that previous work by other researchers had not succeeded in doing. Importantly for such safety-critical systems, the method also yields an assessment of the confidence that may be placed in the optimised ROC curves. The paper results from a collaboration with NATS and a current KTP project, also with NATS, is deploying the methods in air-traffic control centres nationwide.Many safety related and critical systems warn of potentially dangerous events; for example, the short term conflict alert (STCA) system warns of airspace infractions between aircraft. Although installed with current technology, such critical systems may become out of date due to changes in the circumstances in which they function, operational procedures, and the regulatory environment. Current practice is to "tune," by hand, the many parameters governing the system in order to optimize the operating point in terms of the true positive and false positive rates, which are frequently associated with highly imbalanced costs. We cast the tuning of critical systems as a multiobjective optimization problem. We show how a region of the optimal receiver operating characteristic (ROC) curve may be obtained, permitting the system operators to select the operating point. We apply this methodology to the STCA system, using a multiobjective (1+1) evolution strategy, showing that we can improve upon the current hand-tuned operating point, as well as providing the salient ROC curve describing the true positive versus false positive tradeoff. We also provide results for three-objective optimization of the alert response time in addition to the true and false positive rates. Additionally, we illustrate the use of bootstrapping for representing evaluation uncertainty on estimated Pareto fronts, where the evaluation of a system is based upon a finite set of representative data